Automatic Evaluation of Search Engines with Social Relevancy Rank Factoring

In the past precision of information retrieval systems has been evaluated based on general subject queries and some specific query domains both manually and automatically. Manually evaluating the effectiveness of information retrieval systems, in terms of relevance, requires a large amount of human effort and time. Automatic evaluation is much better in adapting to the fast changing web and search engines, as well as the large amount of information on the Web. Many relevance scoring methods have been developed to evaluate the relevance of hits returned by search engines.The future of search almost certainly involves social networks, social graphs, or social filtering in some capacity[12]. The key question is how to organize the data in social media to be used as a factor in relevance scoring. We propose to develop a new measure for evaluating information retrieval systems called the Social Relevancy Rank (SRR). Using the Social Relevancy Rank in relevance scoring, search results will be reordered based on social relevancy to improve retrieval performance. We evaluate this approach throughly using 25 student evaluators from diverse backgrounds for a total of 1250 queries for recall and precision metrics.We believe that the Social Relevancy Rank can be as prominent as Page Rank in a few years. By factoring the social relevancy rank during evaluation of search engines we are contributing towards more efficient and effective real time search and live search in the Web Search Industry. Also, the distinction between real time search,semantic search and social search will diminish and become meaningless. All will together play a role in contextualising and personalizing search for the users and would holistically improve what we think of today as search.